REPORT  DOCUMENTATION  PAGE 


Form  Approved 

OMB  No.  0704-0188 


The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including 
suggestions  for  reducing  this  burden,  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson  Davis  Highway, 
Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of 
information  if  it  does  not  display  a  currently  valid  OMB  control  number. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


5c.  PROGRAM  ELEMENT  NUMBER 

62788F 


5e.  TASK  NUMBER 

PO 

5f.  WORK  UNIT  NUMBER 

LY 


12.  DISTRIBUTION  AVAILABILITY  STATEMENT 

Approved  For  Public  Release;  Distribution  Unlimited.  This  report  is  the  result  of  contracted  fundamental  research  deemed 
exempt  from  public  affairs  security  and  policy  review  in  accordance  with  SAF/AQR  memorandum  dated  10  Dec  08  and 

AFRL/CA  policy  clarification  memorandum  dated  16  Jan  09. 

13.  SUPPLEMENTARY  NOTES 

©2012  IEEE.  Proceedings  International  Joint  Conference  on  Neural  Networks  (UCNN),  Brisbane,  Australia.  10-15  June 
2012.  This  work  is  copyrighted.  One  or  more  of  the  authors  is  a  U.S.  Government  employee  working  within  the  scope  of 
their  Government  job;  therefore,  the  U.S.  Government  is  joint  owner  of  the  work  and  has  the  right  to  copy,  distribute,  and 
use  the  work.  All  other  rights  are  reserved  by  the  copyright  owner. _ 

14.  ABSTRACT 

Memristors  have  been  rediscovered  recently  and  then  gained  increasing  attentions.  Their  unique  properties,  such  as 
high  density,  nonvolatility,  and  recording  historic  behavior  of  current  (or  voltage)  profile,  have  inspired  the  creation  of 
memristor-based  neuromorphic  computing  architecture.  Rather  than  the  existing  crossbar-based  neuron  network 
designs,  we  focus  on  memristor-based  synapse  and  the  corresponding  training  circuit  to  mimic  the  real  biological 
system.  In  this  paper,  first,  the  basic  synapse  design  is  presented.  On  top  of  it,  we  will  discuss  the  training  sharing 
scheme  and  explore  design  implication  on  multi-synapse  neuron  system.  Energy  saving  method  such  as  self-training  is 
also  investigated. 

15.  SUBJECT  TERMS 

Memristor;  synapse;  training 

17.  LIMITATION  OF  I  18.  NUMBER 
ABSTRACT  OF  PAGES 

UU  6 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std.  Z39.18 


19a.  NAME  OF  RESPONSIBLE  PERSON 

NATHAN  MCDONALD 

19b.  TELEPHONE  NUMBER  ( Include  area  code) 

N/A 


16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT 

b.  ABSTRACT 

c.  THIS  PAGE 

u 

u 

u 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Polytechnic  Institute  of  NYU 
2  MetroTech  Center 
Brooklyn,  NY  11201 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Air  Force  Research  Laboratory/Information  Directorate 
Rome  Research  Site/RITB 
525  Brooks  Road 
Rome  NY  13441-4505 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

N/A 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 

AFRL/RI 

11.  SPONSORING/MONITORING 
AGENCY  REPORT  NUMBER 
AFRL-RI-RS-TP-201 3-01 9 


5d.  PROJECT  NUMBER 

T2NC 


3.  DATES  COVERED  (From  -  To) 
nt)  DEC  2010 -NOV  2012 

5a.  CONTRACT  NUMBER 

FA8750-1 1-2-0046 

5b.  GRANT  NUMBER 

N/A 


1 .  REPORT  DATE  (DD-MM-YYYY)  2.  REPORT  TYPE 

MAY  2013  CONFERENCE  PAPER  (Post 

4.  TITLE  AND  SUBTITLE 

MEMRISTOR-BASED  SYNAPSE  DESIGN  AND  TRAINING  SCHEME 
FOR  NEUROMORPHIC  COMPUTING  ARCHITECTURE 


6.  AUTHOR(S) 

Hui  Wang,  Hai  Li,  and  Robinson  Pino 


WCCI  2012  IEEE  World  Congress  on  Computational  Intelligence 
June,  10-15,  2012  -  Brisbane,  Australia 


IJCNN 


Memristor-based  Synapse  Design  and  Training  Scheme 
for  Neuromorphic  Computing  Architecture 


Hui  Wang  and  Hai  (Helen)  Li 
Department  of  Electrical  and  Computer  Engineering 
Polytechnic  Institute  of  New  York  University 
Brooklyn,  New  York,  USA 
hwang07 @students  .poly .  edu,  hli@poly.edu 


Robinson  E.  Pino 

Air  Force  Research  Laboratory,  Information  Directorate 
Advanced  Computing  Architectures 
Rome,  New  York,  USA 
robinson.pino@rl.af.mil 


Abstract — Memristors  have  been  rediscovered  recently  and  then 
gained  increasing  attentions.  Their  unique  properties,  such  as 
high  density,  nonvolatility,  and  recording  historic  behavior  of 
current  (or  voltage)  profile,  have  inspired  the  creation  of 
memristor-based  neuromorphic  computing  architecture.  Rather 
than  the  existing  crossbar-based  neuron  network  designs,  we 
focus  on  memristor-based  synapse  and  the  corresponding 
training  circuit  to  mimic  the  real  biological  system.  In  this  paper, 
first,  the  basic  synapse  design  is  presented.  On  top  of  it,  we  will 
discuss  the  training  sharing  scheme  and  explore  design 
implication  on  multi-synapse  neuron  system.  Energy  saving 
method  such  as  self-training  is  also  investigated. 

Keywords  -  memristor;  synapse;  training 

I.  Introduction 

The  neuromorphic  computing  architecture  that  requires  a 
large  volume  of  memory  and  being  adaptive  to  environment 
has  demonstrated  great  potentials  in  developing  high 
performance  parallel  computing  systems  [1].  Currently,  most 
of  research  activities  have  been  conducted  at  software  or 
system  level  built  upon  the  conventional  Von  Neumann 
computer  architecture  [2]  [3].  Developing  the  neuromorphic 
architecture  at  chip  level  by  mimicking  the  biological  system 
is  another  important  direction.  However,  it  results  in  high 
design  complex  and  cost  by  using  the  traditional  CMOS 
devices. 

Though  the  existence  of  memristor  was  predicted  in  1971 
[4],  the  first  physical  realization  was  first  reported  thirty  years 
later  by  HP  Lab  through  a  Ti02  thin- film  device [5]. 
Afterwards,  more  materials  with  memristive  properties  have 
been  reported  or  rediscovered.  The  unique  properties  of 
memristor  make  it  very  promising  to  be  used  to  mimic  natural 
neuron  networks  [7].  First,  the  memristor-based  memory  can 
achieve  an  integration  density  as  high  as  10  Gbits/cm2  [4]  [5]. 
Second,  the  memristor  device  has  an  intrinsic  and  remarkable 
feature  called  “pinched  hysteresis  loop”,  that  is,  the 
memristance  relies  on  the  total  electric  charge  flowing 
through  it  [4]  [6].  Third,  memristance  remains  unchanged 
when  power  is  turned  off. 


Many  memristor-based  circuit  designs  have  been 
explored,  such  as  crossbar  nonvolatile  memory  [8]  and  FPGA 
[9].  Strukov  et  al.  integrated  digital  memory,  programmable 
Boolean  logic  circuit  and  also  neuron  networks  within  a  3D 
hybrid  CMOS/memristor  structure.  Rajendran  et  al  proposed 
memristor-based  programmable  threshold  logic  array  [11] 
utilized  it  in  synapse-neuron  structure  [12].  However,  the 
training  circuit  and  training  scheme  for  memristor-based 
reconfigurable  architecture  design  have  not  been  fully 
explored  yet. 

In  this  paper,  we  proposed  a  single  memristor-based 
synapse  structure  and  the  corresponding  training  circuit 
design.  On  top  of  it,  we  discussed  the  design  optimization  and 
its  implementation  in  multi-synapse  systems.  With  the  aid  of 
sharing  training  circuit  and  self-training  mode,  the 
performance  and  energy  can  be  significantly  improved.  In  the 
paper,  we  demonstrate  the  effective  of  the  proposed  synapse 
design  by  using  Ti02  memristor,  though  the  design 
philosophy  can  be  generalized  to  other  memristor  materials. 

II.  Preliminary 

In  1971,  Professor  Chua  predicted  the  existence  of  the 
fourth  fundamental  circuit  element,  named  as  memristor,  that 
uniquely  builds  the  relationship  between  the  magnetic  flux 
( cp )  and  the  electric  charge  ( q )  passing  through  the  device  as 
[4]: 

d(p=M'dq.  (1) 

Considering  that  magnetic  flux  and  the  electric  charge  are 
integrals  of  voltage  (V)  and  current  (I)  over  time, 


Figure  1 :  TiCU  thin-film  memristor.  (a)  Structure,  (b)  Equivalent  circuit. 
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respectively.  Eq.  (1)  can  be  expressed  as 
(V  =  M(a>, /)  •/ 

1  dco/dt  —  f(co,I)  ’  ^  ' 

where,  M  (co,  I)  represents  instantaneous  memristance  which 
varies  over  time  and  cois  a  state  variable. 

Figure  1(a)  illustrates  the  conceptual  view  of  Pt/Ti02/Pt 
structure:  two  metal  wires  (Pt)  construct  the  top  and  bottom 
electrodes,  and  a  thick  titanium  dioxide  film  is  sandwiched  in 
between.  A  perfect  Ti02  structure  has  a  natural  state  as  an 
insulator.  However,  the  conductivity  of  oxygen-deficient 
titanium  dioxide  (Ti02_x)  is  much  higher.  By  moving  the 
doping  front  under  proper  electrical  excitations,  the 
intermediate  memristive  state  can  be  achieved.  We  use  RH 
and  Rl  to  denote  the  total  resistance  when  a  Ti02  memristor 
is  fully  undoped  and  doped,  respectively.  The  overall 
memristance  can  be  equivalent  as  two  serially-connected 
resistances,  as  shown  in  Figure  1(b).  That  is 

M(a)  =  a-RL  +  (l-a)-RH.  (3) 

Here,  a  (0  <  a  <  1)  is  the  relative  doping  front  position, 
which  is  the  ratio  of  doping  front  position  over  the  total 
thickness  of  a  Ti02  device. 

For  demonstration  purpose,  we  built  our  design  based  on 
Ti02  thin-film  memristor  and  adopted  the  device  parameters 
from  [5].  The  memristance  range  is  set  from  T^IKQ  to 
R#=16KT1  Note  that  the  proposed  can  be  applied  to  many 
memristor  materials  with  different  physical  mechanisms. 

III.  The  Principle  of  Memristor-based  Synapse 

Rather  than  using  memristor  crossbar  array  in  neuromor- 
phic  reconfigurable  architecture,  we  propose  a  memristor- 
based  synapse  design  to  mimic  the  biological  structure. 
Figure  2(a)  depicts  the  conceptual  scheme,  which  simply 
consists  of  a  NMOS  transistor  (Q)  and  a  memristor.  When  the 
input  Vin  is  low,  Q  is  turned  off  and  the  output  Vout  is 
connected  to  ground  through  the  memristor.  On  the  contrary, 
when  Vin  is  high  and  turns  on  Q ,  memristance  M  and  the 
equivalent  resistance  of  Q  ( Rq )  together  determine  Vout : 

Vout  =  f  (Vin  '  AO-  (3) 

Here,  Vout  is  weighted  by  the  memristance,  which 
behaves  like  a  synapse.  Figure  2(b)  shows  the  simulated  Vout 
when  sweeping  the  memristance  from  1KQ  to  16KQ.  Here, 


Figure  2:  (a)  Proposed  synapse  design,  (b)  Synapse  output  vs.  memristance. 


(a) 


(b) 


Figure  3 :  (a)  The  training  circuit  diagram,  (b)  The  proposed  synapse 
togteher  with  training  circuit. 


CMOS  devices  used  TSMC  0.18pm  technology. 

Note  that  the  response  of  the  synapse  design  is  dependent 
on  the  equivalent  resistance  of  the  transistor  Q  (Rq),  or,  the 
size  of  Q.  This  can  also  be  demonstrated  in  Figure  2(b)  by 
sweeping  the  width  of  Q  from  220nm  to  4.4pm  with  a  step  of 
220nm.  The  simulation  shows  that  a  larger  Q  can  result  in  a 
wider  range  of  Vout  with  poorer  linearity.  However,  for  a  large 
Q,  the  enhancement  of  Vout  by  further  increasing  its  size  is 
marginal. 

To  improve  design  stability,  a  buffer  can  be  added  at 
output  of  the  synapse  to  increase  voltage  swing.  Furthermore, 
some  circuit  optimization  techniques,  such  as  asymmetry  gate 
in  other  blocks,  can  be  used  to  minimize  the  overall  synapse- 
based  system,  as  we  shall  show  in  Section  IV. 

IV.  Synapse  Training  Circuit 
A.  Synapse  Training  Circuit  Design 

Being  self-adaptive  to  the  environment  is  one  of  the  most 
important  properties  of  a  biological  synapse.  To  accomplish 
the  similar  functionality,  a  training  block  is  needed  in  the 
memristor-based  synapse  that  can  adjust  its  memristance. 

1)  Memristor  Training  Circuit 

Figure  3(a)  shows  the  diagram  of  training  circuit  for  one 
synapse  design,  based  on  logic  analysis  and  simplification.  It 
includes  two  major  components:  training  controller  and  write 
driver.  By  comparing  the  current  synapse  output  Vout  and  the 
expected  output  Arain,  training  controller  generates  the  control 
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TABLE  I.  Training  Circuit  Operation  Conditions. 


E 

vout 

Dtrain 

Vtop 

vbot 

17 

r  mem 

Status 

0 

X 

X 

Floating 

0 

X 

Operating 

1 

1/0 

1/0 

0 

0 

ov 

No  training 

1 

1 

0 

1 

0 

1.8V 

Rh  to  Rl 

1 

0 

1 

0 

1 

-1.8V 

Rl  to  Rh 

*  ‘O’-  logic  low;  ‘  1  logic  high,  and  ‘X’  -  unknown  or  don’t  care. 


We  tested  the  training  procedure  by  using  the  Ti02 
memristor  model  [5].  The  training  circuit  was  designed  by 
using  TSMC  0.18jrm  technology  with  FDD=1.8V.  Changing 
memristance  from  RH  to  RL  or  verse  vice  takes  about  51ms. 
The  simulation  result  is  shown  in  Figure  4(b).  Here,  the 
memristance  is  initialized  as  M=16KD.  In  the  first  51  ms,  it  is 
trained  to  1KD  by  setting  Dtrain  to  low.  Then  at  t=51ms,  we  set 
Dtrain  to  high  and  train  the  memristance  back  to  RH  in  the 
following  51  ms. 

2)  Asymmetry  Gate  Design 


TABLE  II.  Sizing  of  INViAndQi 


P/N  Ratio 

PMOS/NMOS  in  INVX 

Qx 

2 

720nm/360nm 

440nm/220nm 

1 8  x  220nm 

16  x  220nm 

1 

360nm/360nm 

1 2  x  220nm 

220nm/220nm 

1 1  X  220nm 

0.5 

360nm/360nm 

220nm/440nm 

9  x  220nm 

9  x  220nm 

signals.  The  write  driver  uses  these  signals  to  control  two 
pairs  of  NMOS  and  PMOS  switches  and  supply  training 
voltage  pair  Vtop  and  Vhot.  The  training  pair  is  applied  to  the 
two  terminals  of  the  memristor  in  the  synapse  design. 

Determined  by  the  training  enable  signal  E ,  the  training 
circuit  can  work  under  two  modes. 

•  Operating  mode :  When  E=0 ,  the  synapse  is  under  regular 
operating  (read)  mode,  and  the  training  circuit  is  disabled. 

•  Training  mode :  The  training  circuit  is  enabled  when  E=l. 
By  comparing  the  current  synapse  output  Vout  and  the 
expected  Dtrain,  the  training  circuit  generates  Vtop  and  Vbot 
applied  to  the  two  terminals  of  memristor  to  update  or 
keep  its  memristance.  We  define  Vmem=Vtop-Vbot. 

The  training  operation  conditions  are  summarized  in 
Table  I. 

Figure  3(b)  depicts  the  proposed  memristor-based  synapse 
integrated  with  training  circuit.  An  extra  NMOS  transistor  Q2 
is  inserted  in  synapse  to  isolate  training  operation  from  other 
voltage  sources:  when  E=1 ,  Q2  is  turned  off  so  that  the  two 
terminals  of  memristor  are  controlled  only  by  the  training 
circuit,  not  affected  by  V-m. 

The  timing  diagram  of  training  circuit  is  demonstrated 
Figure  4(a).  Before  a  training  procedure  starts,  a  sensing  step 
is  required  to  detect  the  current  Vout  to  be  compared  with  Dtrain. 
In  the  sensing  phase ,  accordingly,  training  enable  signal  E  is 
set  to  low  for  a  very  short  period  of  time  ( e.g .,  4.5ns)  at  the 
beginning  of  training.  At  the  same  time,  Vout  is  sent  to  Latch, 
whose  output  Vout'  remains  constant  during  one  training 
period,  as  shown  in  Figure  3(a).  In  the  training  phase ,  E  is  set 
back  to  high  for  a  much  longer  time  ( i.e .,  51ms)  to  change  the 
memristance  if  needed. 


As  we  mentioned  in  section  III,  the  size  of  g7  affects  the 
range  of  Vout.  Instead  of  adding  buffer  or  having  giant  g7  in 
synapse,  the  asymmetry  gate  design  can  be  adopted  to 
minimize  the  layout  area  of  synapse  design.  More  specifically, 
we  tuned  P/N  ratio  of  INV1  in  the  training  circuit  (see  Figure 
3(a)).  Table  II  summarizes  the  required  sizes  of  INV i  and  Qx 
under  different  combinations  that  can  make  training  successful. 
The  result  shows  that  the  asymmetric  design  with  P/N  ratio 
=0.5  can  obtain  the  smallest  area.  The  last  option  is  used  in  the 
following  synapse  analysis. 

3)  Multi-synapse  Training  Scheme 

Most  of  the  neuron  systems  are  constructed  by  multiple 
synapses.  In  this  section,  we  discuss  the  corresponding 
training  scheme  by  taking  a  2-synapse  neuron  in  Figure  5  as 
the  example.  Here,  x47  and  A2  are  two  synapse  inputs  received 
from  other  neurons.  M7  and  M2  are  memristor-based  weights 
for  two  synapses  S7  and  S2.  N  is  denoted  for  neuron  with 
output  Vout.  The  value  of  Vout  depends  on  the  functionality  of  N 
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Figure  4:  (a)  The  timing  diagram  of  training  circuit,  (b)  The  simulation 
result  of  memristor  training. 


3 


Figure  5:  Two-input  neuron  structure. 


Figure  6:Training  sharing  distribution  circuit. 


as  well  as  Voutl  and  Vout2  from  the  two  synapses.  With  the 
different  combinations  of  Mj  and  M2,  the  two-input  neuron 
could  obtain  different  functionality. 

To  save  design  cost,  memristances  of  the  2-synapse  can  be 
trained  separately  and  share  one  training  circuit.  Figure  6 
shows  a  training  sharing  distribution  circuit,  which  generates 
training  signals  to  control  M7  and  M2.  The  training  sharing 
circuit  operations  under  different  conditions  are  shown  in 
Table  III. 

The  two  synapse  inputs  A  7  and  A 2  can  be  used  to  determine 
which  memristor,  M7  or  M2,  is  in  training.  Table  IV  lists  the 
required  A  7  and  A2 ,  when  the  logic  functionality  of  N  is  one  of 
the  following:  OR/NOR,  XOR/XNOR,  AND/NAND. 

Compared  to  the  separated  training  circuit  for  each 
memristor,  the  shared  scheme  can  reduce  26%  of  training 
circuit  transistor  count.  More  saving  in  cost  and  area  can  be 
obtained  when  utilizing  this  training  sharing  distribution 
scheme  to  multi-synapse  structure  with  more  inputs. 

4)  Self-Training  Mode 

To  improve  training  time  and  reduce  power  consumption, 
we  introduce  the  concept  of  self-training  in  our  design:  rather 
than  using  a  fixed  long  training  period  (i.e.,  51ms),  the  self- 


TABLE  III.  Training  sharing  circuit  operation 


Status 

Vtopl 

Vbotl 

Vtopl 

Vbot2 

Training  M7 

vtop 

vb0, 

Floating 

0 

Training  M2 

Floating 

0 

Vtop 

Vbot 

TABLE  IV.  Synapse  Input  pairs  for  different  logics 


Functionality  of  A 

Training  M2 

Training  M2 

OR/NOR 

Aj=1,A2=0 

Aj=0,A2=1 

XOR/XNOR 

Aj=1,A2=0 

Aj=0,A2=1 

AND/NAND 

Aj=1,A2=1 

Aj=1,A2=1 

TABLE  V.  Synapse  Power  Consumption  Analysis 


Operation 

Power 

Energy 

Rl 

1.04  mW 

4.68  pJ 

Read 

Rh 

1 13.4  uW 

0.51  pJ 

From  Rh  to  RL 

216.7  uW 

11.1  uJ 

Training 

Fromi^  to  RH 

234  uW 

11.9  uJ 

training  mode  automatically  stop  programming  memristor 
whenever  Vout  and  Dtrain  become  same. 

The  proposed  training  circuit  supports  self-training  mode 
by  dividing  a  long  training  period  into  multiple  shorter 
periods  and  detecting  Vout  in  between.  The  programming 
period  needs  to  be  carefully  selected:  if  it  is  too  short,  the 
delay  and  energy  overheads  induced  by  Vout  detection  may 
overwhelm  the  benefit  of  self-training.  On  the  contrary,  a 
long  programming  period  cannot  show  enough  benefit. 

The  simulation  result  in  Figure  7  shows  the  memristance 
changing  when  sweeping  programming  period  from  5.1ms  to 
51ms  in  10  steps.  Obviously,  the  self-training  mode  could 
significantly  reduce  training  time.  In  theory,  the  proposed 
training  circuit  can  train  the  memristance  to  any  value  between 
Rh  and  Rl.  The  real  training  time  is  determined  by  the  specific 
application  and  neuron  functionality. 

5)  Power  Analysis 

Reading  and  training  simulation  are  conducted  and  power 
consumption  data  is  collected  in  Table  V.  Energy  is  obtained 
when  setting  read  time  and  write  time  as  4.5ns  and  51  ms, 
respectively. 

V.  Conclusion 

Memristor  has  been  proven  as  a  promising  device  in 
neuromorphic  architecture  for  its  high-density,  nonvolatility, 
and  unique  memristive  characteristic.  In  this  paper,  we 
proposed  a  memristor-based  synapse  that  can  be  used  in 
neuromorphic  computing  architecture.  The  corresponding 
training  operations  including  multi-synapse  schemes  and  self¬ 
training  have  also  been  explored  and  discussed.  The  proposed 
synapse  design  can  be  generalized  to  other  memristor 
materials  for  more  applications.  Next,  we  plan  to  further 
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explore  and  utilize  the  analog  properties  of  the  proposed 
synapse  and  develop  the  memristor  synapse  based  neuron 
network. 
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